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ABSTRACT 


By testing a sample of 1624 Alberta school 
children from the Edmonton Separate and the Stettler 
school districts, the author examined the practical 
limitations of the use of an optically Scored test 
answer sheet. ‘The experimental design used was a 
two-way analysis of variance. 

In one dimension students in grades two to 
nine inclusive were tested using a set of achievement 
tests which were constructed by the author. In the 
other dimension, three types of instruction were 
considered. 

The findings indicated that students in grades 
two and three, in the fall of the year, were unable to 
make valid responses using this technique. Students in 
grades four and five were able to make valid responses 
only after receiving specific instructions and a 
practice session. Above grade five students made valid 
responses regardless of the type of instructions they 
received. Further a set of instructions on the use of 


optically scored answer sheets is included. 
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CHAPTER [I 
INTRODUCTION 


In a testing situation, a systematic procedure is 
followed to allow a tester to compare the behaviour of 
two or more persons, or to measure their achievements or 
objectives. Irrespective of the concept that is being 
considered, the data gathering technique that is being 
utilized by the tester, should in no way bias the decision 
that is made. 

If the systematic procedure used is a multiple 
choice test that requires the testee to respond on a 
separate answer sheet, could this technique in any way 
systematically bias the decision to be made? The author 
has noted that primary school children seem to be unable 
to make valid responses on an answer sheet. If this 
inability is a function of maturation then practice and 
instruction will not facilitate the use of answer sheets. 
On the other hand, if a student's inability to make 
responses on an answer sheet is in fact a function of his 
familiarity with the technique, then a student's performance 
would change markedly if he were taught how to use this 
technique correctly. Thus it is the purpose of this thesis 
firstly, to delimit the grade level and the amount of 
instruction necessary for students in Alberta schools to be 
able to make valid responses using an answer sheet and 


secondly, to find the types of errors are made by students at 
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differing grade levels in order to Suggest corrective 
measures which could be taken to avoid such errors in 
the) future: 

For this purpose permission was gained to test 
students in three schools. Two schools were in the 
Edmonton Separate School system, St. Alphonsus and St. 
Edmunds, and the third was the Stettler Elementary and 
Junior High School. The Edmonton sample was tested in 
the week of October 19th and the Stettler sample was 


tested in the week of November 2nd. 


Importance of the Study 
An Operational Research Bulletin, dated October 1968 


and published by the Department of Education of the 
Province of Alberta pointed out that optical scoring of 
tests and questionnaires was and is availabie to Alberta 
school systems and educational organizations. However, 
the bulletin failed to outline to the potential users of 
this service either the grade level at which valid data 
could be gained using this technique or what kiuds of 
instructions would be necessary to keep the number of 
systematic errors made by the students at a minimum. 
Further, the author has had personal experience 
with a number of research projects in the field of education 
wherein answer sheets were used without the proper safe- 
guards. This is not to suggest that any studies using 


this technique and familiar to the author, jeopardized 


““ 


7 "5 i tas 7 


' at 


: ee ee pee i. ae 
yisoettog Tzeyee'-O2 ““ebto Li» afeval “aba 


-_ 
) 


é | : rau a | 


ssorre dows bbave OF nedes ad btaoa ‘Hat 


JF 
St 
—— 


f er? ey w TOC eaeLerTeqa j 
‘ 
' ’ - 
1 " yi d 2 
, ro ' #4 . 
; I } é a i? é } 
4 ‘ - + 
‘ 
_* 7 \ 
bly or 
m3 ‘De ; ‘ Nie Fe. LS 
bs * 
e,/mawe J 
—. ‘4 4 
: hk eae 
‘Wel G 
a : 
“ay : - 7 
’ oh 3 ’ saved, are | , 
} 
: ‘ J t +4 as 
=" se 
. tTrrond ¢ 
a= oh “ 
— 
ne bol b 
4 ~~ Pp © \ sa 
. cay 29 adnodtes 
ws 
= : 7 
enoilarinagto [stot tSae- tae 
a 7 
7 _—s 


; ’ 
70 etq Laitustoq sit-07 enb ise of behige altel 


ish bilav dotdw ts levek st 14 ‘out vous ts ondeael 
wt 
o abutA ishv, Tosi ernnged re ated ‘gates h ani 


Se seduve siroes s+ hehehe adie é d 

| MR TR tS ohirte sat a r o: 
a ase eiragxe iamoe tig: ‘hs ees " 
elite’ to Stes oat ot 293 avis 


ae erie ee 
gfe 


a ae 


. 


their findings, but rather the answer sheets required 
visual scanning and extensive "cleaning up" before they 
could be scored. Perhaps with a detailed set of 
instructions for both the test administrator and the 
testees the validity of students' responses could be 
insured and at the same time the clerical work necessary 
to prepare the papers for scoring, be reduced to a 
minimum. Future researchers using this technique would 
be advised not to attempt to use it below the lower limits 
as defined by this study both in terms of grade level and 
instruction. 

In summary, this study proposes to provide 
researchers with an adequate set of instructions for the 


use of answer sheets. 


Limitations of the Study 


Since it is the intention of the author to 
generalize to students in Alberta by grade levels rather 
Ean by age, no reference to age has‘been° made. Nor is it 
the intent of this study to classify students by sex, 
although there may be differences in students' abilities if 
they were in fact classified in this way. 

As a result of a pilot project, students in grade 
one were not tested as it was evident that the grade two 
students experienced considerable difficulty in using this 


technique. 


Also the results will be generalized only to answer 
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sheets that can be scored by the IBM-1230 optical scorer. 


Definition of Terms 


Answer sheet. Since the Faculty of Education of 
the University of Alberta, Edmonton and the Department of 
Education of the Province of Alberta, both have IBM-1230 
optical scorers, the term answer sheet, in this thesis, 
will refer to an IBM-5056, Deparment of Education, General 
Purpose Answer Sheet I (example given in appendix) or in 
general any answer sheet that the IBM-1230 optical scorer 


is capable of scoring. 


Answer set. For each multiple choice test item 
there is a set of four alternative answers given. Each set 


being labelled a, b, c, and d, in the test booklet. 


Answer spaces. For each multiple choice test item 
there is a corresponding set of five answer spaces provided 
on the answer sheet. Each set of answer spaces are 


labelled A, B, C, D and E and 1, 2, 3, 4 and 5. 


Score. The total number of correct responses made 
by one individual on a particular set of items, each item 


being worth one, is the score for that individual. 


Mark. This will be the physical mark made on the 


answer sheet by the student responding to a question. 
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Scoring. Herein, scoring will mean to process 


an answer sheet through the IBM-1230 optical scorer. 


CHAPTER II 
SOME RELEVANT LITERATURE 


Although separate answer sheets to be used with 
paper and pencil tests were introduced into the field of 
testing years ago, it was not until recently that machine 
scored answer sheets came into being. Previously the 
teachers were burdened with hand scoring each child's 
test responses. It would seem that as long as a child's 
testeScore. is not invalidated, the adoption of this 
technique in the school system would be highly desirable. 

For a long time test publishers and educators 
alike have held the general opinion that primary aye 
children are unable to use an answer sheet adequately. 
Curiously, research on this question is alrost nonexistent. 

Hieronymous (1961) using the Iowa Test of Basic 
Skills with third graders, found no significant difference 
between three methods of teaching a child to use an answer 
sheet. He concluded that for a normal range of ability 
and tests with a traditional format, third graders can use 
a separate answer sheet effectively. 

Culhane and Stobola (1967), using a questionnaire 
and IBM mankegende cards with students in grade one through 
grade éight, examined the question '... can children at the 
lower elementary school level follow the necessary marking 


instructions with reasonable accuracy?" In their study, 
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the children at each grade level were made familiar with 
the answer sheet and the correct marking of an answer on 
the mark-sense card. By considering double marks and 
light marks, they concluded that mark-sense cards are 
practical for the listing of pupil responses with children 
as low as grade one. They pointed out that if more 
explicit instructions were given as to light marks and if 
additional practice sessions were given at the lower grade 
levels, a large number of errors could possibly have been 
eliminated. 

By giving students a practice session the day prior 
to their actual test, McKee (1967) trained third grade 
students to use an answer sheet. A sample test of five 
items requiring an answer sheet was used to give the 
students practice with the technique. Although this 
experiment was not carried out with all due respect, for 
established research practices, McKee found that the 
practice session was successful in eliminating children's 
anxiety and confusion about the use of answer sheets. It 
was stated that this procedure of familiarizing the students 
with technique should be made a standard practice in the 
future. Furthermore, the author felt it may be desirable 
tovuse a parallel procedure to help orient students of any 
grade level to use an answer sheet for the first time. 

Cashen and Ramseyer (1969) also viewed the question 


of the use of separate answer sheets by primary school 
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children. Their procedure was to have the students 
respond to the California Test of Mental Maturity Short 
Form, Level 1, 1963; using two answering techniques. In 
one administration the students were required to answer 
using the test booklet answer form; and in the other, the 
students answered on a separate answer sheet. The order 
of treatments were randomly assigned to each class involved. 
Their results showed clearly that the test booklet marking 
format resulted in significantly higher performance than 
the separate answer sheet format at both the grade one and 
grade two levels. At the grade three level pupils seemed 
to perform about as well with a separate answer sheet as 
they did with the booklet. The authors further suggested 
that perhaps with formal instruction in the use of answer 
sheets, the grade ones and twos would be able to make 
valid responses. 

In summary, Hieronymous tested third grade students 
of a normal range of abilities with a test of a traditional 
format and found that they could use a separate answer 
sheet effectively. Culhane and Stobola, using mark-sense 
cards, found that after familiarizing the students with the 
answer sheet and the correct marking of an answer that mark- 
sense cards were practical for testing of pupil responses 
with children as low as grade one. By giving third graders 
a practice session, McKee was successful in eliminating 


childrens! anxiety and confusion about the use of an answer 
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sheet. Cashen and Ramseyer found that the use of 
separate answer sheet format resulted in a significantly 
lower performance at the grade one and two levels when 


responding to the California Test of Mental Maturity. 
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CHAPTER III 


ADMINISTRATIVE PROCEDURE, TESTS AND 
INSTRUCTIONS USED 


Test Construction 

The task set for the students was such that the 
decisions made about the usefulness of an answer sheet may 
be generalized to any tests which made use of an answer 
sheet. Typically such a testing procedure would contain 
several components: the conception of the task to be used 
to measure the students' behavior, the administration of 
the task, the communication of a response of the testee 
to the tester, the scoring of the response, and the 
subsequent evaluation of the testee's response by the 
tester. The element of this procedure that will be the 
central topic of concern of this thesis will be the 
usefulness of an answer sheet as a technique of communica- 
tion of a student's response to a tester. 

The means of examining this construct could be 
illustrated by an example: In a multiple choice test using 
an answer sheet, a student is asked, "What is the arithmetic 
sum of 1 + 1?'' The student "knows' the answer to be two, 
and makes a mark on the answer sheet. When the answer 
sheet is scored and if the student receives credit for 
this response, this measure could be said to be a valid 


measure of his ability to add 1+ 1. But if the mark on 
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the answer sheet is such that the student does not 
receive a credit for his response to that question, then 
this could be said to be an invalid measure of his 
ability to add 1+ 1. If the tester is interested in the 
student's arithmetic ability, the means of communicating 
the answer should not systematically bias the actual 
construct that is being considered. 

To make a decision regarding the validity of data 
gathered using an answer sheet, a set of tests of forty 
questionsin length including two types of items was developed 
by the author. For one set of items it was assumed that all 
students within a grade level knew the correct answer. To 
provide a context for these items, the second set of test 
items was made more difficult. The actual responses of 
the students to this second set of items was not considered 
to be relevant to this study. The first set of items was 
embedded within the second set. 

To check the validity of the assumption that all 
students within a grade level did, in fact, know the answer 
to these test items, a parallel treatment, A, was admin- 
aeered to a class of students at each grade level. Rather 
than have the students respond using an answer sheet, the 
students were requested to respond within the test booklet. 
These test responses were subsequently transferred manually by 
the author to an answer sheet, and were scored and analyzed to 


determine the question that had a low difficulty. The 
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criterion for classifying an item as having a low 

difficulty was that not more than one student in the A 

treatment at a grade level failed to respond correctly. 

At each grade level, seven items met this criterion. The 

tests used are included in Appendix A. The specific 

items that were used as the dependent variable are marked. 
The responses of the students in the other treat- 

ments (B, C and D) to this set of items provided the 

dependent variable for this thesis. The independent 

variables used were the grade levels two to nine inclusive, 


and three types of instructions. 


Test Instructions 

The two dimensions to be considered in this study 
are the grade level of the students and the amount of 
instruction they received. 

In treatment B in the Stettler sample, students 
were given a test booklet and an answer sheet. All the 
instruction these students received was, "Indicate the 
answer by placing a mark between the guidelines as shown 
in the example. Use an HB pencil."' This was the minimum 
amount of instruction given in any treatment (see Appendix 
B). , 

In treatment B in the Edmonton sample, the second 
set of instructions (Appendix C) was adapted from a set 
used by Bicknell and Nyberg (1970) in a study for Gibe 


Department of Education, of the Province of Alberta (see 
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Appendix H). This set of instructions included several 
general remarks and suggestions as to the proper use of 
an answer sheet. 

The third set of instructions (Appendix D) was more 
specific. This was a rewording of the instruction set 
provided by the Division of Educational Research Services, 
the University of Alberta, Edmonton, Canada, for the use of 
answer sheets (see Appendix G). In both the Edmonton and 
Stettler samples this set of instructions was used for 
treatments C and D. 

In treatment D, a practice test was given in which 
each of the concepts that were referred to instructions 
were illustrated. Thus, students in this treatment 
received both the detailed set of instructions and a 
practice set (see Appendix E). 

In the Instruction dimension there is a gradient in 
the amount of instruction and practice received by the 
students from a minimum of instruction and no practice in 
treatment B to a maximum of detailed instruction and a 
practice session in treatment D. By using such a range of 
instruction, differences in abilities could be more easily 


identified. 


Administrative Procedure for the Tests 
After randomly assigning each class to a treatment, 
pamphlets were made up to contain the Instructions to 


Administrators, Instructions for the Test, Administrators 
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Copy of the Test, test booklets and answer sheets. The 
instructions and test items were read to the students in 
grades two to four for all treatments; above grade four, 
only the instructions were read to the students (see 
Appendix F). 

For treatments A, B and C, the pamphlets were 
distributed to the teachers concerned to administer. 
Within each school and at each grade level, treatments 
A, B and C were administered at one time by the classroom 
teachers. 

Subsequently, the author administered treatment D, 
which gave the students practice in using the technique 
by responding to a set of sample questions. For each 
grade level the practice test was parallel in form and 
content to the main test. Also, the class worked through 
Gnerpracticertest! togetheréselectingithe correct jalter- 
native. The author and the home room teacher circulated 
within the class to check to see that each student was in 
fact able to make an answer in the correct manner. 

This practice test, written immediately prior to 
the main test, was used to familiarize the students with 
these concepts: 

(a) For each test question there was given an 
answer set labelled a, b, c and d on the test booklet. 
Corresponding to each answer set there was a set of answer 


spaces labelled A, B, C and D. For each question the 
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students were instructed to decide which answer was 
correct and then to make a mark in the corresponding 
answer space. 

(Db) “Each set of answer spaces was also numbered 
1, 2, 3, 4 and 5. The students were instructed to 
disregard these labels in marking their response to a 
question. 

(c) The required size, shape and density of a 
mark was pointed out. 

(d) The answer spaces proceed columnwise, thirty- 
five to a column. 

In summary, the task was a set of achievement 
tests of forty questions in length and containing two 
types of items. One set of test items was set so that 
each student at a grade level was assumed to know the 
correct answer. The second set of items provided an 
overall task for the students of an achievement test. 

A set of instructions was also developed. This set of 
instructions varied from nonspecific to a very specific 
set with a practice session. Further, the administrative 


procedure followed by the author was outlined. 
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CHAPTER IV 
PILOT STUDY 


In the week of October Sth, 1969, seven heterogeneous 
classes of students from the Edmonton school system were 
selected for a trial test administration. Table 1 shows 
the specific number of students and schools involved. As 
there were only four classes available for testing in the 
main study at grades three and six, no classes were 


available for the pilot study from these grade levels. 


TABLE 1 
SAMPLE SIZES IN PILOT STUDY 


rr eS he ener 


Grade Name of the School Number of 
Tested Students 
2 St. Alphonsus 23 
2 St. Edmunds 25 
4 St. Alphonsus Z5 
5 St. Edmunds 24 
7 Donnan Junior High aut 
8 Donnan Junior High 30 
9 Donnan Junior High 28 
Total 186 


By personally administering the set of tests, the 
author was able to practice the administrative procedure for 


treatment D, and at the same time note any difficulties the 
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students had in taking the test. 

Subsequently, the answer sheets were scored and 
punched on cards for analysis. The data were used to 
check for any changes in the abilities of students to 
respond using answer sheets over the grade levels included 
in the pilot study. For the particular set of instructions 
used, it was noted that grade two students had difficulty 
in making valid responses, whereas grade four students and 
older seemed to make more valid responses. 

For each grade level, the Division of Educational 
Research Services (1969) item analysis program was used to 
help point out fiaws in the test set. Primarily, 


typographical and clerical errors were detected. 
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CHAPTER V 
MAIN STUDY 


Statistical Procedures Used 

For the explicit purpose of this study there was a 
set of test items the answers to which were assumed to be 
known by each testee for a given grade level. By analyzing 
the responses made to this set of items, it was possible 
to examine a general underlying assumption in testing that 
the person being tested is capable of communicating his 
perceived answer to the scorer in such a form that the 
scorer is able to evaluate, by implication, what the 
testee perceived as being the correct answer. At each 


grade level there were seven items in this "known" group. 


‘Model to be Used (Winer, 1962) 

The model used was a p x q completely fixed factorial 
design where factor p is grade level with grades two to 
nine inclusive being considered, and factor q is type of 
instructions being given. Three sets of instructions were 


used. 


Underlying Theoretical Model 
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Verbal Hypotheses 
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population mean. 

differential effect due Corgrade 1. 
differential effect due to treatment j. 
differential effect due to an inter- 
between grade i and treatment j. 


individual difference component for 


k. 


H.: The data gained by means of an answer sheet 


from students in each of grades two to nine 


are valid. 


1! The data gained by means of an answer sheet 


from students in each of grades two to nine 


are invalid. 


H.: The validity of data gained by means of an 


answer sheet for each grades two to nine is 


not affected by the type of instruction given. 


1: The validity of data gained by means of an 


answer sheet for each of grades two to nine is 


affected by the type of instructions given. 


Model Type 


Grade levels of students is a fixed factor since 


this study will only generalize to the particular grade 


levels studied. 


Also levels of instruction is a fixed 


factor since this study will only generalize to the 
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particular types of instructions used. 


Underlying Assumptions of Analysis of Variance 


(a) Variance due to individual differences within 
each of the treatment populations is 
homogeneous. 

(b) Treatment effects are additive. 

(c) The observed scores are based on an interval 
scale of measure. 

(d) The experimental errors are independently 
and normally distributed within each of the 
pq populations with a mean of zero and common 
variance of oe, 

Treatment of the Data 

The observed data were tabulated in 2 table with 
eight rows (factor p) and three columns (factor q). By 
considering the computational symbols, a summary of the 
analysis of Daten: was computed using ANOV 25 of the 

Division of Educational Research Services (1969). The 

expected values required for the analysis of variance were 


as given in Table 2. 
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TABLE 2 


COMPUTATIONAL SYMBOLS USED IN COMPUTING 
AN ANALYSIS OF VARIANCE 


a 


Source of Degrees Mean Expected R 
Variation of Squares Mean Ratio 
Freedom Squares 
Main effect 2 2 
of A (p-1) ms , choogs o, ms,+ ms, cell 
Main effect 2 2 
of B (q-1) MS), g_+6 o, msp+ ms, cell 
Interaction 2 9 
of AB (p-1) (q-1) MS op o.+3 Sig MSapt mS, cell 
chee 2 
Within cell qp(n-1) ms 
w cell oe 


Total npq-1l 


ANOV 25 carried out a two-way analysis of variance with 
unequal observations per cell assuming fixed effect model. 
Firstly, the additive model (2) is tested against a more 
general model applying the least square techniques to test 


the interaction effect. 


Sie Seg ee Rags eae (1) 
igiahor De pepions injesch Geil rangeiit (2) 


If the interaction effect is found to be not significant, 
model (1) is used assuming no interaction. If the inter- 


action is found to be significant, model (2) is used. 
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Statistical Hypothesis Regarding the Main Effects 
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Decision Role 

If the F observed is greater than the F critical, 
reject Hy and accept Hy; where the critical value of F is 
given by: 

A: Fy_, [(p-1), (-paq)] 

B: F,_, [(a-1), (N-pq)] 

C: Fi), [(p*1)(q-1), (-paq)] 
Here the number of persons in each cell ranged from twelve 
to thirty-one, the number of grade levels (p) considered 
was eight, and three types of instructions (q) were used. 


The F's were considered for 2's of 0.01. 
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The Edmonton Sample 
In the week of October 19th, 1969, sample classes 


were drawn from two Edmonton schools, St. Alphonsus and 

St. Edmunds. These heterogeneous classes were assigned 
randomly to treatments. Table 3 shows the number of 

students involved. The tests were administered following 

the procedure as outlined in the section of this thesis 
entitled Administrative Procedure for the Test. Subsequently, 
the dependent variable scores were calculated. The data 
handling procedures for both the Edmonton and Stettler 


samples are described below. 


TABLE 3 
NUMBER OF STUDENTS IN EDMONTON SAMPLE 


Treatment A B C D 
Grade 

Ze 24 26 Za, 29 
3 24 24 Le 25 
4 30 25 28 24 
5 26 24 ZL is 
6 26 Z5 ol 29 
vi 24 26 28 ob 
8 24 ue 25 22 
9 30 29 28 29 


To be able to comment on the validity and reliability 


of the data handling procedure two parallel and independent 
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procedures were followed (see Figure 1). The three 
branches of this figure will be outlined. In each case 
the procedure begins with the raw data. 

In branch one, the answer sheets were scored, 
keyed for the seven questions that comprised the dependent 
variable. TEST 04 of the Division of Educational Research 
Services (1969) was used to check that the questions in 
treatment A that constituted the dependent variable did 
meet the criteria of low difficulty. Then ANOV 25 of the 
Division of Educational Research Services (1969) was used 
to make the statistical decisions and to provide an 
estimate of cell means and cell frequencies. 

In branch two, all the forty items were scored and 
TEST 04 of the Division of Educational Research Services 
(1969) was used to determine the questions in treatment A 
that did meet the criteria of having a low difficulty. 
Then TEST 06 of the Division of Educational Research 
Services (1969) was used to rescore treatments 8, C and D 
keyed only for the seven questions that constituted the 
dependent variable. ANOV 25 of the Division of Educational 
Research Services (1969) was used to make the statistical 
decisions and estimate the cell means and cell frequencies. 

In the third branch, the data were hand scored, 
keyed to the seven questions that constituted the dependent 
variable. This provided an estimate of the cell means and 


cell frequencies. 
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Cell Means 


TEST 04, TEST 06, and ANOV 25 were programs of the 


z5 


DATA 


Hand Scored Keyed 
to dependent 
variable 


All 40 items 
scored 


E 


Test 04 used to 
determine 

Questions with a 
low difficulty 
in treatment A 


Test 06 used to 
determine 
dependent vari- 
able in treat- 
ment B, C and D 


ANOV 25 
Statistical 
Decisions 


Cell Cell 
Frequencies Frequencies 


Cell Means 


Division of Educational Research Services (1969). 


FIGURE .1 


FLOW CHART OF THE PROCEDURAL CHECK 
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The cell means and cell frequencies derived in 
each branch were compared to see that they agreed within 
acceptable limits. Exact agreement was not expected 
since the machine's scoring does not have a reliability 
of one. Since the cell means and cell frequencies did in 
fact agree within acceptable limits this provided a check 
on the transferring of the data in treatment A to the 
answer sheets. The ANOV 25 of the Division of Educational 
Research Services (1969) used in branch one and two 
provided a check on the statistical decision. 

In summary, by considering the reliability of three 
separate estimates of the cell means and cell frequencies 
and two estimates of statistical decisions, the author 
checked the reliability and validity of the data handling 


procedure used in the study. 


Analysis of Variance 


The test of additivity showed the interaction term 
to be significant so model (2) was tenable. This indicated 
that students at different grade levels beneficed 
differently from the type of instruction they received. 

In considering the main effects a weighting system 
was used. Since this study intends to generalize to Alberta 
schools by grade, the samples by grade were weighted 
proportionally to the numbers of school children enrolled 
in that grade as reported by the Department of Education of 


the province of Alberta Annual Report 1968 (see Table 4). 
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These were the most recent statistics available. 


TABLE 4 


DISTRIBUTION OF PUPILS BY GRADE AS OF 
SEPTEMBER 30, 1967 IN ALBERTA 


Grade Number of Pupils Proportion 

of Total 

2 58,277 On S92. 

3 36,715 Oreos os 

4 36 ,084 0.13124 

5 34,408 0.12514 

6 33,403 0.12149 

7 34,239 OFZ 453 

8 52 too Orel 17 Or 

9 29,491 Owl OF26 


The analysis of variance of main effects indicated 
that there was a significant grade effect, a significant 
instruction effect, and a significant interaction (see 
Table 5). The grade effect means that students' ability 
to use an answer sheet changed from grades two to nine. 

The instruction effect means that the students' performance 


changed considering three types of instruction. 
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TABLE 5 
ANALYSIS OF VARIANCE FOR THE EDMONTON SAMPLE 


(This solution is based on weighted main effects 
and tested without the additivity assumption) 


SSS a a 


Source Sum of Degrees Mean F Probab- 

Squares of Square Ratio JaeLes 
Freedom of F 

Main effects 

of Grades 361.97 7 S71 40.71 0.000002 

Main effects of 

Instruction 20716 2 10.08 7.93 0.00040 

Interaction 

effect of 

Instructions 

and Grades 49.03 14 3.50 ahs 80.200 05i7 

Within Cell 774. 80 610 Valk 


To aid in interpreting this change in performance, 
Scheffé multiple comparisons of grade level means was 
performed. The results were tabulated in Table 7, using 
the weighted means as tabulated in Table 6. It can be seen 
from this table that the performance of the grade two 
students differed significantly from all other grades. Also 
grade three students differed significantly in performance 
from grades six, seven, eight and nine; and grade four 


students differed from grade nine. 
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TABLE 6 


WEIGHTS AND MEANS USED IN THE SCHEFFE TEST FOR Tak 
EDMONTON SAMPLE 
SSE 


Weights used for levels of factor A (grades): 


Grade 2 3 4 . 5 6 7 
Weight 

Used ORS 921 20. 153555 Fadals1 ad Oc 2514°°0.12149 0.12453 
Grade 8 9 

Weight 


Used Ovd76de"0510726 


Means by Grades: 


Grade 2 3 4 5 6 7 8 9 
Weighted 
Main Effect 420° 0.06. 6.23..°6.355.).6. 80" 9Oer7 6.87 16.87 


Weight used for levels of B (treatments): 


Treatment B C D 
Weight Used 0.33333 OF So555 0.33334 
Means by Treatments: 

Treatment B C D 


Mean 6.02 6.30 6.46 


TABLE 7 


SCHEFFE MULTIPLE COMPARISONS OF MEANS BY GRADES FOR THE 
EDMONTON SAMPLE 


Grade 
Grade 2 3 4 5 6 Z 8 9 
r Fs FS k & * ® 
0. OZ 04 035+" 0 017 02% 006 
OF 00 0206 


2 
3 
4 
5 
6 
fi 
8 
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*» 0.001 
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In the other dimension when typ2 of instructions 
were considered, it was noted that the performance under 
instruction set 1 was different from the performance 


under instruction set 2 and set 3 (Table 8). 


TABLE 8 


/ 
SCHEFFE MULTIPLE COMPARISONS OF MEANS BY TREATMENTS 
FOR EDMONTON SAMPLE 


B C D 
B 0.04 0.0005 
C 0.361 


Since the interaction term in the analysis of 
variance proved to be statistically significant, it was 
necessary to analyze the simple main effects. In the one 
dimension, it was of interest to see if for each grade 
level the mean criterion scores differed over the type of 
instruction given. Profiles of simple main effects were 
graphed by plotting the mean criterion score versus treat- 
ment classification by grade level (see Figures 2 and 3). 
In the other dimension it was of interest to see if for 
each treatment the mean criterion scores differed across 
grade levels. Profiles of simple main effects were 
graphed (Figure 4) by plotting the mean criterion score 


(Table 9) versus grade level by treatment. 
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a Grade 2 7 Grade 3 
§ a 
5 5 
4 “a 4 
3 3 
: 2 
i 1 
0 0 
B G ps B G D 
Treatment Treatment 
7 Grade 4 7 Grade 5 
6 LE ee 2 6 
5 3 | 
4 4 
5 3 
2 
0 
Be once D B C D 
Treatment Treatment 
FIGURE 2 


SIMPLE MAIN EFFECTS FOR THE EDMONTON SAMPLE BY 
GRADE FOR GRADES 2 TO 5 
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SIMPLE MAIN EFFECTS FOR THE EDMONTON SAMPLE BY 
GRADE FOR GRADES 6 TO 9 
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SIMPLE MAIN EFFECTS FOR THE EDMONTON SAMPLE BY - 
TREATMENTS 
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. TABLE 9 


CELL MEANS FOR THE EDMONTON SAMPLE BY TREATMENT 


2 HO02e: 3.42 4.96 She 
3 7.00 Seal 6.32 6.16 
4 7-00 6.00 5.935 Sy 72 
= 7.00 6.12 6.26 6.65 
6 7.00 6.76 6.84 Gmt7 9 
A 7.00 6.88 oye el 6.71 
8 100 Oau2 6.84 6.86 
9 7.00 6.86 6. 86 6.90 


From the graphs of the simple main effects ty 


betaces,, it can. be seen that at the grade levels two to 
five there was a trend in the means from low to high from 
treatments B to D. That is, the numbers of errors made 
when using an answer sheet were least in treatment D and 
greatest in treatment B. Further it was noted that the 
largest differences occurred at the lower grade levels. 

But when considering the graphs, Figure 3 of the 
simple main effects for the grades six to nine, the 
numbers of errors made seemed to be independent of the 
type of instructions the students received. 

When considering the simple main effects of grades, 
within each treatment it was noted that there was a trend 


in the numbers of errors made by students being greatest 
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at grade two and least in grade nine. The single largest 
difference being noted between grades two and three (see 
Figure 4). 

In summary, the analysis of variance indicated that 
grade two and three students were unable to make valid 
responses at an acceptable level regardless of the type of 
instruction given. Grade four and five students were able 
to make valid responses when given a detailed set of 
instructions and a practice session. Students 4s grades 
above five, made valid responses regardless of the type of 
instructions they received. 

A tabulation of the specific SCE of errors made 
by the students, is shown in Table 10. Other possible 
sources of confusion when students were making responses 
were: 

(a) The grade twos were confused in the change 
from lower case letters in the designation of the answer 
set to the upper case designation of the answer spaces. 

(b) In the arithmetic subtest, although the answer 
sets were labelled a, b, c ona d, students often responded 
in answer spaces numbered the same as the correct answer, 
rather than by the letter designation of the answer space. 

The noe common type of errors were multiple marks 
and poor erasures. So in administering a test using an 
answer sheet, students should be given specific instructions 


in this regard. Further, for this scoring machine light 
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TABLE 10 
ERRORS MADE BY STUDENTS WHEN RESPONDING 


Grade Treatment Multiple Poor Others Sum 
Marks Erasure 

2 B Ss 6 5 25 
C 11 £S 9 53 

D 12 13 0 25 

5 B 6 8 1 ike 
C 2 0 0 2 

D 5 iz 1 8 

4 B 4 0 0 £ 
e 5 4 0 9 

D 0 0 0 0 

5 B 1 0 0 1 
C 1 0 0 L 

D 1 2 0 3 

6 B 5 0 0 S) 
C 1 u 0 2 

D iu il 0 2 

7 B és 1 0 5 
C 2 1 0 <) 

D ib 0 0 al 

8 B 0 0 0 0 
C “9 0 0 5 

D 0 0 2 2 

i) B al 0 0 1 
C 0 2 0 2 

D 2 0 0 2 


Others included: (in descending order of frequency). 


1. Poorly made marks. That is, marks that were too light, 
long, short, or thick. 


2. The students were noted to have either omitted a 
question or an answer space, thereby completing 
part of the test incorrectly. 

3. The students were noted to proceed into Part Leon 
the answer sheet to respond to question Sooracne. 
than going to the top of the second column of answer 
spaces. 
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marks did not seem to be a problem. 


The Stettler Sample 


In the week of November 2nd, 1969, classes in the 
Stettler elementary and junior high school were assigned 
to treatments randomly. These classes were homogeneously 

grouped by ability. Therefore at each grade level type 
of instruction was confounded with ability grouping. 
Table 11 shows the numbers of students involved at each 
grade level. The same administrative procedure was 
followed as in the Edmonton sample, and the dependent 
variable scores were calculated using the same questions 


for the dependent variable. 


TABLE I 


NUMBERS. OF STUDENTS IN THE STETTLER SAMPLE 


2 a 14% on ots 
3 725 24 31 14% 
4 30 24 18* 29 
5 as 23 15% za 
6 33 29 20* 25 
7 20 ot oa i 
8 - 30 12% a 29 
9 25 26 24 28 


*These classes are in a four year remedial program. 
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Analysis of Variance 


The test of additivity showed the interaction term 
to be significant. So model (2) was tenable indicating 
that students at different grade levels benefited 
differently from the type of rea fon they received. 

Again, in considering the main effects, a weighting 
System was used to weight the samples proportionally to 
the numbers of school children enrolled in that grade in 
1968 in Alberta schools. This analysis (Table 12) 
indicated that there was a significant grade effect, and 
interaction effect, but the instruction effect proved to 
be non-significant having a probability of 0.045. As in 
the Edmonton sample students' abilities to use an answer 


sheet changed from grade two to eight. 


TABLE . 12 
ANALYSIS OF VARIANCE FOR THE STETTLER SAMPLE 
(This solution is based on weighted main effects 


and tested without the additivity assumption) 
SSS SSS SSS ee eS. 


Source Sum of Degrees of Mean F Probabil- 
Squares Freedom Squares Ratio ities 

Grades 

(A) 86.51 is 12.36 14.43 0.000003 

inst ruc- 

tions (B) 5.35 2 Zi. Oo! Sebe 0.045 

S (AB) 34.77 14 Ppa Nel a OKA is 

S(E) 457.48 534 0.86 
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A Scheffé multiple comparison of means was done 
considering the students grade levels. The results were 
tabulated in Table 14 using the weighted means as tabulated 
in Table 13. It can be seen from the table that the 
performance of the grade two students differed significantly 


from all other grade levels. 


TABLE 13 


WEIGHTS AND MEANS USED IN THE SCHEFFE TEST FOR THE 
STETTLER SAMPLE 


Weights used for levels of factor A (Grades): 


Grade z 3 4 5 6 
Weight 

Used 0513921. —0.13353 0.13124: 0.12514. .0.12199 
Grade 7 ; 8 9 

Weight 

Used 0.12453 0.11761 0.10726 


Means by Grades: 


Grade 2 3 4 S 6 7 8 9 
Mean 556, 6.43 6.507 6.75 6.76. -6.80° (6:79 76:87 


Weights used for levels of factor B (Treatments): 


Treatment B C D 
Weight Used Ks) 0.33333 0.33354 


Means by Treatments: 


Treatment B aah D 
Mean 6.49 6.40 6.64 
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TABLE 14 


SCHEFFE MULTIPLE COMPARISONS OF MEANS BY GRADES FOR 
THE STETTLER SAMPLE 


————— == reste 


Grade 2 3 4 5 6 7 8 9 

Grade 

Z * 0.004 * * * k — 

5 

4 

5 

6 

7 

8 

9 
SSS a SS 
* p > 0.001 


In the treatment dimension, it was noted that the 
remedial classes tended to have lower cell means than the 
other students at that grade level irrespective of the 
instructional treatment they received. Thus there was a 
confounding effect of ability by treatment resulting in 
no significant difference in the instruction dimension at 
piece tas" 0.01 levelz. @But at theta = 0.05¢level of Sagni- 
ficance, the instruction effect was significant as a 
observed = 0.045. 

Since the interaction term in the analysis of vari- 
ance proved to be statistically significant, it was necessary 
to analyze the simple main effects. In the one dimension, 
it was of interest to see if for each grade level the 
criterion scores differed over the types of instructions 
given. As for the Edmonton sample, profiles of simple main 


effects were plotted (see Figures 5 and 6). But because of 
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the confounding of the treatment with the ability of the 
students, interpretation of the effect of treatment by 
grade was difficult. However in examining the simple main 
effects by grades within each treatment a trend was noted 
from lowest to highest from grade two to nine (see Figure 
7). Also the single largest difference was noted between 
grades two and three. Table 15 contains a summary of cell 


means used in these graphs. 


LADLE seis 


CELL MEANS FOR THE STETTLER SAMPLE BY TREATMENT 


SSS SSS SSS SSS SSS See 


Treatment A B G D 
Grade 

2 EU Grea 4.85 5.60 
3 7.00 6.29 6.64 6.50% 
4 7.00 6.04 6.00% 6.86 
5 7.00 6.65 6.60* ¥00 
6 100 6. 86 Guo0 6.92 
7 1.00 Gh. S57 On 6.90 
8 7.00 6. 58% 7.00 65,79 
9 SAY OF ok 6.96 6.86 


* Classes in a four year remedial program. 


In summary, for the Stettler sample the analysis of 
variance indicated that grade two students were unable to 
make valid responses at an acceptable level regardless 
of the type of instruction given. Grade three, four and 
five students were able to make valid responses when given 


a detailed set of instructions and a practice session. 
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Mean Criteria Score 


vi Grade 2 ? Grace 3 
6 6 ———— 
= a E 
4 4 
3 3 
2 2 
1 1 
0 0 
Be «G D B C D* 
Treatment Treatment 
7 Grade 4 7 Grade 5 
6 — Ss 6 
5 > 
4 4 
3 3 
7 2 
1 1 
0 0 
B Gs D B Ge D 
Treatment Treatment 


Class in four year remedial program. 


FIGURE 5 


SIMPLE MAIN EFFECTS FOR STETTLER SAMPLE BY 
GRADE FOR GRADES 2 TO 5 
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- Class in four year remedial program. 


FIGURE 6 


SIMPLE MAIN EFFECTS FOR THE STETTLER SAMPLE BY 
GRADE FOR GRADES 6 TO 9 
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* - Class in a four year remedial program. 


FIGURE 7 | 
SIMPLE MAIN EFFECTS FOR THE STETTLER SAMPLE BY TREATMENTS 
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Students in grades above five, made valid responses 
regardless of the type of instructions they received. 

The specific types of errors made by the students 
were tabulated in Table 16. Again it was noted the grade 
two's were confused in the change of case of the letter 
designation of the answer set and the answer spaces. 

Also the number designation of the answer spaces caused 
some students difficulty. Again the most common type of 


errors were multiple marks and poor erasures. 


Comparison of the Edmonton and Stettler Samples 


When comparing the B treatments for the two samples 
it should be remembered that after comparing the actual 
instruction sets used in treatments B and C (Appendix C 
and D), for the Edmonton sample, it was decided to alter 
Eneowinstruction set used in the B treatment for the Stettler 
sample to provide a greater degree of variability among 
instructions. Consequently, the Stettler students were 
given a less specific set of instructions for the B treat- 
ment than the Edmonton students. (Appendix B). 

For purposes of comparison, considering the graphs 
of the simple main effects for both samples by grade, 
some important consistencies were evident. When consider- 
ing the graphs (Figures 2 and 5) for grades two to five 
for both samples, the grade twos' and threes' in both 
cases were unable to make valid responses regardless of 


the type of instruction they received. The grade four and 
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TABLE 
ERRORS MADE BY STUDENTS WHEN RESPONDING 


Others Sum 
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Marks Erasure 


Treatment Multiple 
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five students benefited differentially from the 
different types of instructions they received. Namely, 
the students who received the D treatment made valid 
responses whereas the students in treatments B and C 
did not. In grades six through nine in both samples 
(Figures 3 and 6), the responses made by the students 
were valid. 

One atypical class mean was observed, grade two, 
treatment B of the Stettler sample (Figure Se For 
although this was a remedial class and received the 
least amount of instruction, their mean score (6.21) 
was higher than any of the other grade two scores in 
either sample. Considering their ability grouping, it 
is difficult to understand why these students would display 
such a high score when no other remedial class did. 
Perhaps see high score could be due to a nonuniform 
application of the test instructions by the classroom 


teacher. 
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CHAPTER VI 
SUMMARY AND RECOMMENDATIONS 


In considering the results it should be remembered 
that the tests were administered to the students in the 
fall of the year. The grade designation refers to students' 
grade level as of the date of testing. Secondly, the 
conclusions rest on the assumption that the items to which 
group A students responded in a near errorless fashion 
would also have been responded to in an errorless fashion 
by the students in groups B, C and D at that grade level 
if they had responded in the test booklet rather than on 
an answer sheet. 

In the Edmonton sample, the analysis of variance 
indicated that grade two and three students were unable to 
make valid responses at an acceptable level regardless of 
the instructions given. Grade four and five students were 
able to make valid responses when given a detailed set of 
instructions and a practice session. Students in grades 
above five, made valid responses regardless of the type of 
instruction they received. In the Stettler sample, the 
analysis of variance indicated that grade two students are 
unable to make valid responses at an acceptable level 
regardless of the type of instruction given. Grade three, 
four and five students were able to make valid responses 


only when given a detailed set of instructions and a 
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practice session. Above grade five students made valid 
responses regardless of the type of instruction they 
received. 

In terms of the verbal hypotheses being tested the 
data gained by means of an answer sheet from students in 
grade two and three are invalid regardless of the 
instructions given to the students. In grades four and 
five a detailed set of instructions and a practice session 
are required to insure the validity of the data. Above 
grade five, the data gained by means of an answer sheet 
were valid réegardlesssof the type).of, instruction: ..Further, 
by examining the difficulty level of all test items in 
treatments B, C and D, it was concluded that the converse 
conclusion was not tenable. That is, only two items at 
the grade three level and one item at the grade four level 
would support the hypothesis that students answering on an 
answer sheet performed better than the students who 
responded in the test booklet. 

When preparing an examination that uses an answer 
sheet an examiner should bear in mind that younger students 
can be confused by changes in labelling from an answer set 
to an answer space. Specifically, some students at the 
lower grade levels were confused when the answer set was 
labelled in lower case letters and the answer spaces were 
labelled with capital letters. Further, on the arithmetic 


subtest, some students responded in answer spaces numbered 
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the same as the correct answer rather than by the letter 
designation of the answer space. 
On the basis of the study a number of recommend- 


ations to prospective users are possible. 


Recommendations for the Instructor 

(a) Although blank answer sheets can be overprinted 
with a non-reflecting ink, answer sheets duplicated or 
printed by other means will not meet the specifications of 
tolerance for the IBM-1230 optical scoring machine and 
thus cannot be scored by an optical scorer. 

(b) Ensure that neither the student nor the examiner 
place any marks of any kind among the timing lines (the 
black lines in the right-hand margin of the answer sheet). 
Watch particularly to ensure that the personal information 
such as the name of the school and the name of the test 
are not run into the timing marks. 

(c) Decide in advance what additional information 
you will require for each student and, whenever possible 
have the student code this data on the answer cheet at the 
time of testing. 

(d) Make certain that each student has a sharp HB 
(medium soft) pencil for marking answers. 

Do Not Use: (1) an IBM electrographic pencil, 
(2) wean Hoe 2H, SH.s etc pencil, 
(3) a pen, 
(4) a felt tipped marking pen. 
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Above grade seven, the examiner should provide 
pencils to students for examination purposes. 

(e) When storing answer sheets keep them flat. 

Do not fold, staple, or bend any of the corners. 

(f) If each student is to receive a set of answer 
sheets, it is best to staple these together on the lower 
left hand corner of the answer sheet. This is because 
the top of the answer sheet is the leading edge that goes 
into the optical scorer and should be as free from folds, 


bends or tears as possible. 


Recommendations for the Students 

(a) Do not mark more than one answer to each 
question. The optical scorer will score as wrong any 
question with more than one answer indicated. Stray 
pencil marks, smudges or dots on the guidelines might 
also be interpreted by the machine as "double-answers" 
and cause an item to be marked wrong. 

(b) If you change your mind about an answer, erase 
the unwanted answer completely. Poor or smudged erasures 
may be read by the machine. 

(c) When indicating answers, blacken the entire 
space between the guidelines corresponding to each choice. 
Do not make a mark larger than the guidelines. While the 
scoring machine has considerable tolerance with respect to 


marks that are too long, too short or too light; carelessly 
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made marks increase the chances of a auestion being marked 
wrong by the machine. The most satisfactory marks are 
made by using a sharp pencil and stroking over the guide- 
lines two or three times. A dull pencil makes a mark of 
the correct width with one stroke, however, the lines are 
usually not black enough to score properly every time. 


For example: 


Proper mark -- se! Li (ee) ss E44 
Improper Marks: 

Ale BS eeeCrs. D4) 76S 

too ea oy SSsEe - ESES a BESS SESS 

ght AU eee Cr Sie Dee Emo) 

too short =) = S5cSS (facia) see SES= 0 EES== 

too long as = Boe ae oe a 

LOomthick -- AJ B2 Dud) 65 


In conclusion, by considering the responses of 1624 
Alberta school children in grades two to nine inclusive, 
the author was able to set guidelines, in terms of grade 
level and instruction type, for the use of optically scored 
answer sheets. By complying with the limitations as 
specified, a research worker will be able to gather valid 
data using this technique keeping the number of systematic 


errors made by the students to a minimum. 
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APPENDIX 


This appendix contains qopies of the stést 
used by grade. The questions used for the dependent 


variable marked with an X. 
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1. 
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Look at line number 1, See the balls. Mark the atswer on the answer sheet 


ag. ih bin ie ce ae d. () 


Look at line number 2. See the stars. Mark the answer for the biggest star. 


for the biggest ball. 


a. A b, tA Ce d. ie 


Look at line 3. See the blocks. Mark the answer for the smallest block on 


the answer sheet. 
chads <M eee aa cae | 


Look at line 4. See the moons. Mark the answer for the smallest moon on 
the answer sheet. } 2 } 
a. (i be (: Ce osm d. ) 


Look at line number 5, See the first block. Now mark a. answer on the 
answer sheet for the other block that is just like it. 


BR outed Cuieer cased’ 4alios 


Look at line number 6. See the little window. Find the other window like 
it and mark the answer for the other window on the answer shect. 


ee eet ee cent ce 


ee ee 


Look at line number 7. See the first ball. Now find the other ball like it 
and mark the answer for that ball on the answer sheet. 


Ze Re AG a ee 
AY a G5 & CS C4 OD 
Look at line 3. See the first block. Mark the answer on the answer sheet 
for the other block that is just like it. 


Look at line 9. See the first block. Now mark the answer on the answer 
sheet for the other block that is the same. 
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Look at number 10, See the first figure. Mark the answer on the answer 
sheet for the other figure that is just like it. 
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For questions 11 to 19, mark the answer on the answer sheet for the word that 
is the opposite in meaning of the word underlined. 


iI. 


12. 


The opposite of 
up is 

The opposite of 
black is 


The opposite of 
hot is 


The opposite of 
far is 


The opposite of 
father is. 


The opposite of 
work is 

The opposite of 
happy is 

The opposite of 


ze 


Ge down 


Ge blue 


a. cold 


a. beside 


a, aunt 


a. fun 


a. glad 


a. end 


b, play 


b. green 


b. cool 


b. neor 


b ° mother 


b. ames 


b. gay 


b. lag 


Cc. run 


c. red 


Cc. sun 


c. under 


Cc. sister 


Ce job 


ce sad 


Ce new 


d. see 


d. white 


d. warm 


d. up 


d. uncle 


d. play 


J. joy 


d. old 


For questions 19 to 30, find the answer that makes the following statements TRUE. 


X19, 
X 20. 
745 
225 


l+1 5 
24+2= 
1+2= 
4¢+1]= 
2+6=5 
5+4= 


6+22 


a. 0 


a. 4 


dent. 
a. 2 
a. 0 
a. 0 


b. | 


Ce 2 


ce. 6 


A ence 
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auld a) 


29. 12-11= aa O eb taste d. -3 
30.16 -14= Si Mela ler Getics A d. 4 
31. 


32. 


33. 


Look at line 31. The first three drawings in the row are alike in certain ways. 
Find the drawing at the right that fits in the blank. 


ua og Oe, [r2.| w» tite ml bavlegs al ale AOS 


Look at line 32. The first three drawings in the row are alike in certain ways. 
Find the drawing at the right that fits in the blank. 


os O () p the a. O ». ee: al) 


For the rest of the test, continue to answer the questions as in questions 31 and 32. 
Find the drawing at the right that fits in the blank. 
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X 1. 


ee 


Look at line number 1. See the balls, Mark the answer on the answer sheet 
for the biggest ball. 


Look at line number 2, See the stars. Mark the answer on the answer sheet 
for the biggest star. 


tive ‘a ce 


fe a b. ee Ce AS d. ay 


AY 


Look at line number 3. See the blocks. Mark the answer on the answer 
sheet for the smallest block, 7 


al i Sc ty | hha 


Look now at line 4. See the moons. Mark the answer on the answer sheet 

for the smallest moon. :. ‘ 
f ‘ Py : ( ( 

Qe Sask Be “ib Co ; F 2 d. a Ose 

y oy a = 


Look at line 5. See the first block, Now mark the answer on the answes 
sheet for the other block that is just like it. 
Ne ame a. eae b, pond ati nS d. i _ Re 


Ep ee aL ieee eae eke Be bohiesed 


Look at lie b. See the little ball, Find the other ball like it and mark 
the answer for the other window on the answer sheet. 


oes. atte, 
Vey FET . \ *, ie IN, ee 
{ NS : (oe b./ wy eh ! ‘ de: Sol 
~! Gel ie oe of 1, j Ge ! : of -* 
pd hee ae ee, \ sy . / aes 
’ ee ‘ ee : as 


Look at line 7. See the first window. Find the other window just like it 
and mark the answer on the answer sheet. 


(mea sece, eae ee ae Geelial: {eae | 
j oN: a ' os rab “ 1 f i q } iy ae 
bea sf Ge i 4 b, H Pa 5 Ce i j | d. a ‘/ 
See a Na we ee Peers 


Look at number 8. See the block, Now find the other block just like tf 
and mark that answer on the answer sheet. 
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foom wath 


9. For number 9 do the same. 


10. For number 10 do the same. 
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For questions 11 to 20, have the students mark the answer on the answer sheet 
for the word that is the opposite in meaning of the word underlined. 


11. The opposite of 
up is : 


12. The opposite of 
black is 


13. The opposite of 
het is: : 


14. The opposite of 
far is 


15. The opposite of 
father is : 


16. The opposite of 
work is 


17. The opposite of 
happy is : 


18. The opposite of 

first is : 
19. The opposite of 

boy is : 


20. The opposite of 
short is : 


a. down b. play Ce run d. sce 
a. blue b. green ce red d. white 
a.cold b. cool C. sun d. warm 
a. beside _——b, near c.under d, up 
a.avnt. 6. mother c.sister d. uncle 
a. fun b. games _ c, job d. play 
ae gay b. glad Ce joy d. sad 
a. end b. last Cc. new d. old 
a. brother _b. girl c.euncle d, sister 
a. bunch b. fall ce small = d.. tall 
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For questions 21 to 30, have the students find the answer that makes the following 
statements TRUE. 


21. <ltl= a. 0 b. 1 Cb 2 d. 3 
X22, 2+2= a. 4 b.2 Geo das 
ze 1 +2 = a. 0 b. 1 eee d, 3 

24. 4415 a. 4 b. 5 ce 6 dav 

25. 2+6= a. 6 b, 7 eins ds 9 

26. 5+13= Geis, b. 18 Genie, d. 20 

27, 4445 Ee a. 16 be. 17 c. 18 d. 19 

28. Il-7= a. 3 b. 4 C.0 d. 19 

7h a Ain Men a. 0 b. J Cs d. 3 

30. 16-14= a. 0 b. 1 ce 2 d.3 


For cach of questions 31 to 40, the first three crawings in the row are alike in 
certain ways. Find the drawing at the right that fits in the blank. 


a1, le] [a [ea ae af] b. I] ce C] dee 
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grade 4 
1. 
2. 
60> 
Bee | 
X 5. The opposite of 
up iss 
6. The opposite of 
black is; 
x 7. The opposite of 
hot is: 
8. The opposite of 
far is: 
9. The opposite of 
father is: 
X 10. The opposite of 
work is: 
11. The opposite of 
happy is: 
12. The opposite of 
first is: 
13. The opposite of 
oes 
14. The opposite of 
short is: 


15. The opposite of 


active iss 


Geis 


‘a. down 


a. blue 


a. cold 


a. beside 


a. aunt 


a. fun 


Ge gay 


a. end 


a. brother 


a. bunch 


a. quiet 


be play 


b. green 


b. cool 


b. near 


b. mother 


b. games 
b. glad 
b. last 
b. girl 
b, fall 


b. quilt 


Ce. sun 
c. under 
c. Sister 
Ce» job 
Ce joy 
c. new 
c. uncle 


ce small 


c. quit 
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d. uncle 


d. old 


d.sister 


d. tall 


d. quite 
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16. 1+4= 


7.2+2= 


18. 1+2= 


19.4+1= 


20.2+6= 


21. 12+11= 


22. 37 X4= 


23. 135+ 5= 


24. 17 -7= 


25. 30 - 16= 


26. 503 - 26] = 


27.17 X11= 


23. 45 -15= 


29. pare} {>} 


— 


30. OO) SN ae 
31, i i 


32. O 
xo. | 

as, C) 
@ 


35. @ « 


Oo 


i 


gee 
Se 
= 


Go 


o @ 


Nee 
b,.0000 


nd! 
0 


®e¢e 
Dolelwie 


66 
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For questions 1 to 12, mark the answer on the answer sheet for the word that is the 


opposite in meaning of the word underlined. 


al 


10. 


(PL 


UZ; 


The opposite of 
up is 


The opposite of 
black is 


The opposite of 
hot is 


The opposite of 


far is 


The opposite of 
father is 


The opposite of 
since is 


The opposite of 
active is 


The opposite of 
LOL 


The opposite of 
work is 


The opposite of 
deep is 


The opposite of 
attic is 


The opposite of 
future is 


a. down 


a. blue 


a. cold 


a. beside 


a. aunt 


a. become 


a. quiet 


a. brother 


a. fun 


a. dodge 


a. basement 


a. farther 


b. green 


b. cool 


b. near 


b.mother 


b. beggar 


b. quilt 


b. girl 


b. games 


b. drank 


b. roof 


b. further 


Ce. run 


c. red 


Ce. sun 


Cc. under 


c. sister 


c. before 


Cc. quit 


ce. uncle 


Cc. job 


Ce empty 


Ce room 


d. see 


d. white 


d. warm 


d. uncle 


d. beneath 


d. quite 


d. sister 


d. play 


d. shallow 


d. wire 


d. patent 


ary 7 Ni a) 
on ont Sh ent oR 


bd 
- ny byw ott 40 an i 


Are a¢ 
s ’ ar 
y i 
7 
' [p> 
uJ ley Pe “veo a 
4 
ce ae hold. 
‘i ¥ ) 4 bigs af 
, a 
at gga. 0 
O] 
’ aa 
a > alt 
7 deter foe: 
it . niu Pye 
= . ! ~ 
- 
a Koos 7 
P j ae p 
S | i Smngo — «@ ; ia _ ? 
me AEDT 
a ee 
‘ “i 
. suo so “ ard 
7 >. 7 
OT 
~ a — 
j = ys ©€ 


radtiond ib 


ano pei 
‘a dal 


69 
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For each of questions 13 to 26, find the answer that makes the following statement TRUE: 


13, 1+1= 


14,,2+2= 


15. 1+2= 


16, 4+1]= 


2 6 = 


18. 12#11= 


19.37 X4= 


20, 135 +5= 


21. 17 -7= 


22. 30 - 1é= 


23. 503 = 261 = 


24. 17X11 


25. 


a. 0 


a. 4 


a. 0 


b, | 


b. 5 


b, 232 


Ge 2 


c. 138 
Ce. 29 
c. 24 
c. 24 
c. 242 


c. 187 


Cay. 
20 


d.3 


reli VA 


] it = * °s 
WUWOHOT orf) 29> 
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For each of questions 27 to 40, the first three drawings in the row are alike in 
certain ways. Find the drawing at the right that fits in the blank. 


27.19 q ey ea neh Fee eri 


28. GrG oe 2 a fw ‘ble © Co ma do @ 
ce) 

29. if ya sks nad | Si c. —~o dom 
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ae ke I Oe b,co0o c. oO do 
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GRADE 6 
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For questions 1 to 11, mark the answer on the answer s..cet for the word that is 
the opposite in meaning of the word that is underlined. 


—_ 
s 


9. 


10. 


Il. 


The opposite of 
up is 


The opposite of 
black is 


The opposite of 
hot i 


The opposite of 
far is 


The opposite of 
father is 


The opposite of 
since is 


The opposite of 
active is 


The opposite of 
attic is 


es 


The opposite of 


gobs 


The opposite of 
future is 


The opposite of 


prosperity iss 


a. down 


a. blue 


a. cold 


a. beside 


a. aunt 


a. become 


a. quiet 


a. basement 


a. fun 


a. farther 


a. poultry 


b, play 


b. green 


b. cool 


b. near 


b. mother 


b. beggar 


b, quilt 


b. fool 


b. games 


b. further 


b. poverty 


Cc. run 
c. red 
c. sun 
Cc, oe 
c. sister 
c. before 
Cc. quit 
c. roof 
C. job 
C. pust 


c. priority 


d. see 


d. white 


d. warm 


d. up 


d. uncle 


d. beneath 


d. quite 


d. wise 


d. play 


d. patent 


d. property 


Lb . 
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6-2 
For each of questions 12 to 25, find the answer that makes the following TRUE 
and mark that answer on the answer sheet. 


12, 1+] = a. 0 ee | c.2 d. 3 
1372/42 = a. 4 b. 5 c. 6 ra ier 
4,.1+2= a. 0 b. 1 c. 2 oh | 
15.1+4=5 a. 4 b. 5 c. 6 da Z 
16, 1I+I11= G::2% 6. 29 Ce al d. 36 
17, 528 + 72 = a. 600 b. 605 c. 700 d. 800 
18. 473 = 26= a. 447 b. 457 c. 467 d. 477 
19, 50X 44= a.2200 b. 2300 c. 2400 d. 2500 
20. 6 X 807= . a. 4742 b. 4842 ce. 4942 d. 4952 
21.5-15 a. | b. 2 co 3 d, 4 
é 6 6 6 6 6 
22,.4+1 = a. 17 b, 18 Coal d. 
5 7 20 20 Si 
23.44 -3 55 a. 4 b. 7 ce 16 d. 60 
ra 9 iO) 
24,34 %1 = a. 7 b. 8 c. 13 d. 35 
10 10 2 2 
25 5X8 a. 8 b, 15 c. 8 d. 15 
6 610 & 60 6 Té 
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For each of the question 24 to 40, the first three drawings in the row are alike in 
certain ways. Find the drawing at the right that fits in the blank. 
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GRADE 7 
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For questions 1 to 11, mark the answer on the answer sheet for the word that is the 
Opposite In meaning of the word that is underlined. 


1. 


ya 


?. 


10. The opposite of 


11. The opposite of 


The opposite of 
up is 


The opposite of 
black is 


The opposite of 
hot is 


The opposite of 
far is 


The opposite of 
active is 
The opposite of 


work is 


The opposite of 
since is 


The opposite of 


destruction is: 


The opposite of 
prosperit 


nervous is 


comedy is 


a. down b. play ¢. run d. sce 

a. blue b. green c. red d. white 
a. cold b. cool ¢. sun d. warm 
a, beside b, near ¢. under d. up 

a. quiet b. quilt c. quit d. quite 
a. fun b. games Ce job d. play 

a. become b. beggar c.. before d. beneath 


a. constellation b. constitution c. construction 
d. conversation 


a. priority b, property c. poultry 4. poverty 


a. calm b. normal = c.. nylon d. weird 


a. musical b. parody _c.. satire d. tragedy 
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7-2 
For each of the questions 12 to 25, find the answer that makes the following 
TRUE and mark that answer on the answer sheet. 
12, 1+1= a. 0 b. 1 coe dog 
ice 2+2= a. 4 b,. 5 ce 6 d. 7 
14. 1+2= a. 0 b. 1 pays d.3 
Ie 1+4= a. 4 b. 5 Gee d. 7 
16. 528 +72= a. 600 b. 605 c. 700 d. 300 
17. 473 -26= a. 447 b. 457 ce. 467 d. 477 
18. 50X 44= a. 2200 b. 2300 c. 2400 d. 2500 
9. 4 +1= a. | b. 2 c. 17 d. 18 
5 5 5 20 
20.44 -3 5 = a. 4 b. 7 c. 16 d. 50 
6 6 NOS 
ot. 3. * 3. = a. 8 a. 15 c. 8 d. 15 
6 To 0) 60 16 Té 
22.3. «= a. 3 b. 3 c. 4 d.3 
$77 32 {2 {2 2 
23. 862 = .372= ae -490 b. .590 ce. 490 d. 590 
24, 44X2 = a3 b. 1 co7 d. 18 
3 6 5 5 
4 


20121 = a. 4 be 8 CunlG d. 32 
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For each of the question 24 to 40, the first three drawings in the row are alike in 
certain ways. Find the drawing at the right that fits in the blank. 
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For questions 1 to 11, mark the answer on the answer sheet for the word that is the 
Opposite in meaning of the word that is underlined. 


1. 


2. 


EL 


The opposite of 


up is : 
The opposite of 
black is : 


The opposite of 
hot is 


The opposite of 
far is 


The opposite of 
work is 


The opposite of 
active is 


The opposite of 
prosperity is: 


The opposite of 
nervous is 


The opposite of 
co medy igs 


The opposite of 
destruction js: 


The opposite of 
feminine is : 


a. down b. play c, run 
a. blue b. green c. red 
a. cold b. cool c. sun 
a. beside b. near c,. under 
a. fun b. games c. job 
a. quiet b. quilt c. quit 


a. priority b. property c. poultry 


a. calm b. normal c. nylon 


a. musical b. parady c. satire 


d. play 


d. quite 


d. poverty 


d. weird 


d. tragedy 


a. constellation b. constitution c. construction 
d. conversation 


a, marvellous b. masculine c. merciful d. merely 
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For each of the questions 12 to 25, find the answer that makes the following 
TRUE and mark that answer on the answer sheet. 


12. 1+1= a. 0 b. 1 ore 4 d. 3 
13. 2+2= a. 4 b. 5 c. 6 d. 7 
14, 1+2= a. 0 b. J Cc. 2 d. 3 
15. 1+4= a. 4 b. 5 c. 6 d. 7 
16, 523 +72= a. 600 b. 605 c. 700 d. 300 
17. 473 -26= a. 447 b. 457 c. 439 d. 499 
2. 5 XS = a. 8 b. 15 c. 8 d. 15 
6 fd 60 60 [% r3 
Wisi Se = a, 3 b. 3 c. 4 d. 3 
ee 3 32 2 M2 > 
44-3 5 = a. 4 b, 7 c. 16 d. 50 
6 6 oe “o. 
21. «862 - .372 = a. .490 b. .590 c. 490 d. 590 
29, APX2= a. 3 o7 oil d. 13 
3 5 6 5 
4 
23.2 = a. 16 b. 32 c. 64 d. 128 
Nel Meee 2 = a, 7 b. 14 Ce. Ag d. 49 
16 18 16 4 
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For each of the question 26 to 40, the first three drawings in the row are alike in 
certain ways. Find the drawing at the right that fits in the blank. 
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GRADE 9 


For questions 1 to 11, mark the answer on the answer sheet f or the word that is the 
opposite in meaning of the word that is underlined, 


1. The opposite of 
up is: a. down ob. play” c. run d. see 


X2. The opposite of 
black is: a. blue b. green c. red d. white 


X 3, The opposite of 


hot iss a. cold b. cool cc. sun d. warm 


4. The opposite of 
far is: a. beside b.nmeor c.under d.up 


5. The opposite of 
work is: : a. fun b. games c. job d. play 


6. * The opposite of 
active is: a. quiet b. quilt c. quit d. quite 


7. The opposite of 
rosperity is: a. priority b. propertyc. poultry d. poveriy 
X 8, The opposite of 
comedy is: a. musical b. parody c. satire dd. tragedy 


9. The opposite of 
destruction is: a. constellation b. constitution c. construction 
d. conversation 


X 10. The opposite of 
feminine is: a. marvellous b. masculine c. merciful 
d. merely 


11. The opposite of 
dessert is: a. abandon b. grassland c. hold d. soup 
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For each of the questions 12 to 25, find the answer that makes the following TRUE and 
mark that answer on the answer sheet: 


| Wa 
13. 
14, 
15. 


16. 


1+1= 


2+2= 


14+2= 


Og Ho 
il 


~862 er 3/2 


44X2 = 
3 


Ge 


a. 0 
a. 4 
a. 0 
a. 4 
a. 600 


a. 8 
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8-3 


For each of the question 26 to 40, the first three drawings in the row are alike in 
certain ways. Find the drawing at the right that fits in the blank. 
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(Instruction Set 2) 


A Study of Students' Abilities. 

Instructions for the Test: 

(1) We wish to find out how well you are able to do 
certain kinds of problems. We ask, therefore, that 
you do this test. 

(2) Each question is worth one mark. 

(3) Work quickly. Please do not leave out any questions. 

(4) Mark the space on the answer sheet for each of your 
answers. 

(S) Do all forty quesittions. But if you find a question 
you are unable to answer, leave it and return to it 
if you have time at the end of the exam. 

foleeeete in, at the top of the answer sheet, your: 

(a) Name 
(b) Grade 
(c) Class or Teacher's. name. 
(7) An example is given below: 
Ate Aga 4 (a) ie :7 (b) 8 (co 2 (a) te 
(b) is the correct answer. Now see how this example 


is marked on the answer sheet. 


Do Not Write in This Booklet, Please. 
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(Instruction Set 3) 


Aeoeudy of Students" Abilities. 


Instructions for the Test: 


(1) 


(2) 


(3) 


(4) 


(S) 


(6) 


(7) 


(8) 


(9) 


(10) 


(11) 


We wish to find out how well you are able to do certain 
kinds of problems. We ask, therefore, that you do this 
LeS.c. 

Each question is worth one mark. 

Do not make the marks for the answers larger than the 
spaces between the red lines. 

Mark only one answer for each question. 

To change an answer erase your first answer completely. 
Use a sharp H.B. pencil only (medium soft) 

Do not make any marks in the black lines along the 

Side of the answer sheet. 

Do not fold or bend your answer sheet. 

Do all forty questions. But if you find a question you 
are unable to answer, leave it and return to it if you 
have time at the end of the exam. 

A sample question is given below: 

4+4= Galen? (Byes (c) 9 (d) 10 

(b) is the correct answer. See how this answer is 
marked on the answer sheet. 

At the= top: of your answer sheet, f111 in: your: 

(a) Name 


(b) Grade 
(c) Class or teacher's name. 


Please do not write in this test booklet. 
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PRACTICE TEST FOR GRADES 2 & 3 


1. 


= 
iS) 
fe 
w 
| 


9. The opposite of 
old is: 


10. The opposite of 
up is: 


a. a 


a. /{/ | 


a. new 


a. see 


F. run 


Ge { J 
oor; 

c. | 

Ce 


| d. first 


d. down 


PRACTICE TEST FOR GRADES 4 &5 


ee 


—ee | 


5. 7X12= 
6, 125X5= 


7. The opposite of 
buy is: 


8. The opposite of 
Fix is: 


a. oe b, ES 


a. 19 b. 74 

a. 5 b, 25 
a. bough b. box 
a. brake b. break 


C. 325 


c. girl 


c. broke 


d. 625 


d. sell 


d. repair 


PRACTICE TEST FOR GRADES 6 & 7 


wT SLT 


————— & 


3.12+17= 

4, 476 - 249 = 

5. The opposite of 
least is: 


6. The opposite of 


victory iss 


a.-127 


a. less 


a. defeat 


ol 
b. FH 
b. 28 


b, 227 


b. more 


b. defence 


c. 237 


c emost 


c. defend 


Bae 


d. much 


d. descent 


nw 


wae 4 > ia 
a Bei 1 
ae 
> 
[om 
ri 
» 
ccc 
—_ . oh 29 
, 
4) ’ } 


Sse, 


a ad 


93 
PRACTICE TEST FOR GRADES 8 &9 


Mi Wd WP Loe ok HE aR 
PH tence (|| omelette sa llt 


NO 

e 
=e 

a 


3. 649 - 362 = a. 267 b. 287 c. 367 d. 387 
4. 1/4X4/5= a. 1/4 b. 1/5 c. 5/9 d. 21/20 


5. The opposite of 
victory is: a. defeat b. defence c. defend d. descend 


6. The opposite of — 
majority is: a. nationality b. neutrality c. minority d. municipality 
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appendix contains a copy of: 


Instructions for Practice Test 
Instructions used in treatment A 
Instructions to Administrators 
Administrators’ Instructions, for Grade: 2 
Administrators' Instructions for Grade 3 


Administrators' Instructions for Grade 4 
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INSTRUCTIONS FOR PRACTICE TEST 


To help you understand how to answer the test 
exercise, a practice test will be given. In the test 
that you are about to take, the answers are to be made 
on a separate answer sheet. There is a row of answer 
spaces numbered to match each test question. 

To make an answer, first decide which is the best 
answer. Find the set of the answer spaces numbered the 
same as the question on the answer sheet. Then make a 
black mark in the answer space for the answer you have 
chosen. 

There are three important things to remember: 

(1) Make a heavy mark within the red lines. 

(2) Make only ONE mark for each question. 


(3) Erase any wrong answers completely. 
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OF 
A Study of Students! Abilities. 


Instructions for the Test: 

(1) We wish to find out how well you are able to do 
certain kinds of problems. We ask you, therefore, 
to do this test. 

(2) Each question is worth one mark. 

(3) Do all forty questions. But if you find a question 
you are unable to answer, leave it and return to it 
if you have time at the end of the exam. 

(4) Now please fill in: 

(a) Your name: 
(b) Your grade: 
fc) Wour class: 
(5) Answer by putting an X through the correct answer. 


Example: 4+ 42 (a) 7 (b) K (co) e.g (d)an10 
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Instructions to Administrators: 


It is essential that any teacher who is to administer 


the test entitled "A Study of Students' Abilities" read 


these instructions. 


Purpose: 


The test is in the form of a multiple choice exan, 
which is frequently used to evaluate the abilities of 
pupils. The purpose here, is not to measure directly the 
students' abilities per se but rather to see how well the 
students eer Te to use an optically scored answer sheet. 
The value of the results could be jeopardized if the 


instructions are not followed carefully. 


Administrative Procedure for the Test: 

Make certain that each pupil has: 

(a) A sharp H.B. pencil (medium soft) 

(b) A test booklet 

(c) A piece of scrap paper on which to do rough work. 
Then ask the students to read, with you, the "Instructions 
Lo? tne Lest; 

For grades 5 to 9, then allow the students to complete 
the test. 

For grades 2 to 4, see enclosed administrators 


instructions for the exam. 
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Grade 2 Administrators Instructions: 


For each of questions 1 to 30 inclusive, read each 
question and answer set to the students allowing time for 
the students to respond. If the students have difficulty 
with a particular question tell them to leave it blank 
and return to it later if they have time. 

Do question number 31 as an example. After reading 
the instructions for the question to the students, point 
out that there is a set of three boxes each just a little 
larger than the one before. it. Point out. that (c) is the 
correct answer because it is the box that would fit in 
the blank to complete the set of boxes. 

For questions 32 and 33, read the instructions as 
given and then instruct the students to do questions 34 
to 40 on their own like the last three questions. Remind 
the students to go back and do any questions they left 


out if they have any time at the end of the test. 
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Grade 3 Administrators Instructions: 


For each of questions 1 to 30 inclusive, read each 
question and answer set to the students allowing time 
for them to respond. If the students have difficulty 
with a particular question tell them to leave it blank 
and return to it if they have time at the end of the 
EeSC. 

Do question number 31 as an example. After reading 
the instructions for the question to the students, point 
out there is a set of three boxes each just a little 
larger than the one before it. Point out that (a) is 
the correct answer because it is the box that would fit 
in the blank to complete the set of boxes. 

For questions 32 and 33, reread the instructions as 
given and then instruct the students to do questions 34 to 
40 on their own like the last three. Remind them to go 
back and do any questions they may have left out if they 


have any time at the end of the test. 
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Grade 4 Administrators Instructions: 


For questions 1 to 4, read the following 
instructions: 


(1) Mark the answer for the smallest ball. 

(2) Mark the answer for the largest box. 

(3) Mark the answer for the figure at the right 
a, b, c or d, that is the same as the figure 
at the left. 

(4) Mark the answer for the figure at the right 
a, b, c or d, that is the same as the figure 
at the Jett. 


FOFSGuUeSt10NS: > to 15> .instruct the students to 
mark the answer for the word that is the opposite in 
meaning of the word that is underlined. Allow about 
7 minutes for the students to complete this set of 
questions. If the students have difficulty with a 
particular question, tell them to leave it blank and 
return to it later if they have time. 


For questions 16 to 28, have the students find the 
answer that makes the following statements true. Again 
allow about 7 minutes to complete the set of questions. 


Do queStion 29 as an example. The instructions read: 
The first three drawings in the row are alike in certain 
Ways. Find-the drawing at the right that fits in the 
blank to complete the set. Point out that there is a 
set of -three boxes each just a little larger than the one 
before it. Point out that (c) is the correct answer 
because it is the box that would fit in the blank to 
complete the set of boxes. 


For questions 30 to 40, instruct the students to 
pinlagthne stuouresat. the right a, b,ac or dy that fits” in 
the blank to complete the set. Allow a further 7 minutes 
for the students to complete the test on their own. Remind 
the students to go back and do any questions they left out 
if they have any time at the end of the test. 
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INSTRUCTIONS TO USERS OF IBM-1230 MACHINE SCORED 


ANSWER SHEETS 


it 1s very important that answer sheets to be scored 
on an IBM-1230 scoring machine be marked very carefully. 
Because the scoring mechanism is very sensitive, students 
who try to "fool" the machine or who make marks incorrectly 
may cause the machine to score their paper incorrectly. 


If the instructions given below are followed by the 
students, there should be no errors in scoring their answer 


sheets: 


(2) 


(2) 


(3) 


(4) 


Use a sharp HB pencil, only, for marking answers. 


DO NOT USE - (a) an IBM Electrographic Pencil; 
Oba sanm Be ZH SH, ete. penc1l: 
(c) a pen. 


When indicating answers, blacken the entire 
space between the guidelines corresponding to 
each choice. DO NOT MAKE A MARK LONGER THAN THE 
GUIDELINES. (See examples below). While the 


scoring machine has considerable tolerance with 


respect to marks that are too long, too short or 
too light, carelessly made marks increase the 
chances of a question being marked wrong by the 
machine. The most satisfactory marks are made 
by using a sharp pencil and stroking over the 
guideline two-or three times. A dull pencil 
makes a mark of the correct width with one stroke, 
however, the lines are usually not black enough 
tOusGore properly every time. 


Be careful not to place any marks of any kind 
among the timing lines (the black lines in the 
right-hand margin of the answer sheet). Watch 
particularly so as not to run the personal 
information such as the name of the school and 
the name of the test into the timing marks, nor 
extend the answers past the guidelines in the 
last column of answers adjacent to the timing 
marks, 


DO NOT MARK MORE THAN ONE ANSWER TO EACH QUESTION. 
The machine will score as wrong any question with 
more than one answer indicated. Stray pencil 
marks, smudges or even dots on the guidelines 
might also be interpreted by the machine as 
"double-answers' and cause an item to be marked 
wrong. 
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(5) If you change your mind about an answer, 
erase the unwanted answer completely. Poor 
or smudged erasures may be read by the machine. 
Old erasers that have hardened are often a 


cause of smudges. 


(6) DO NOT FOLD AN ANSWER SHEET NOR BEND ANY OF 
THE CORNERS. 


Below are examples showing a proper mark and several 
improper marks: 


PROPER MARK: ze! %:2 S92 Su! kid 
too light ya! a <3 Dt 5 
too short 2 fe! 922 Crack B..f £.5 
too long 9 fs! Sede Sed Pet fod 
COGENT 4 Aad Qed. fed Bad feck 
wrong tees 8.23: $23 Press eee 


wrong we SSSss 0 SS555 0 @RBBB SESS 
wrong Bee! 8..2 ee ae , 


ecesee 8 sence 


. 


wWeorvren. 
ic ite voce ‘brtim sey oye 
clotelqmes tawana Resmi 
/ bose et wee esrcerte 
siont eved See? eteee 
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A STUDY OF PUPILS" CONCEPTS 


Instructions ror filling in the forms 


We wish to find out how you feel about school and 
some school subjects. We ask you, therefore, to fill in 
the 7 sheets that go with these instructions. 


At the top of each sheet is a title word, such as the 
name of a school subject. Please tell us how you feel 
about the title word by placing a mark between each of 
the 24 pairs of words that follow it. 


You can see that each pair of words is a set of 
opposites. You can therefore tell us how you feel by 
looking at the two words, then marking the space between 
the two that correspond to your answer. An example is 
shown below: 


Title Word - MUSIC 
hard 


casy 


bers eS teresting 

A person who feels that music is very easy, and 
neither interesting nor boring, would mark the spaces 
as shown. 


If you are not taking the subject listed as the title 
word, leave that page blank. 


Wotkesgurchiy., 1t as better to give your first 
feelings rather than to think hard about each pair of 
words. Please do not leave out any of the pairs. 


Your marks will be read by a machine, so feel free 
foetell how,you really feel. 


Do not fill in-any of the blanks at the top of the 
sheet until after you have completed the questionnaire. 
Your teacher will then give instructions as to what 
information is to be given. 
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